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Reply to Final Office Action of July 26, 2006 

REMARKS/ ARGU MENTS 

Prior to this Amendment, claims 1, 3-15, 17-20, 22, 23, and 25-30 were 
pending in the application. No claim amendments are presented with this 
Amendment, and the Listing of Claims is provided only for the convenience of the 

Examiner. 

Finality of July 26, 2006 Office Action 

The application has been subject to three Office Actions and a Request for 

Continued Examination has been filed to continue prosecution in the case. The 

most recent Amendment after the November 4, 2005 Office Action clarified the 

claims mainly by bringing dependent claim limitations into independent claims. 

A new Examiner has been appointed to examine the application and in 
preparing the July 26, 2006 Office Action, the Examiner performed yet another 
search. This has resulted in the July 26, 2006 Office Action citing 2 new references 
(j e ., Freeman and Schuster). However, the Office Action was made final - which, 
of course, places limits on adding or amending claims to address the new citations. 

Applicant, therefore, respectfully requests that the finality of the Office Action 
be reconsidered and withdrawn. Applicant believes if a new Examiner had not been 
appointed it is likely that a new search would not have been performed, in any 
case, the claim amendments presented in the prior Amendment did not present new 
issues that were not before the Office due to the long history of prosecution for this 
application or otherwise require the additional search and new grounds for rejection. 

However, Applicant does appreciate the fact that all anticipation rejections of 
the claims have been withdrawn and only obviousness rejections need to be 
addressed in this Amendment. 


WBO - 082705/000001 - 190109 v1 


9 


AppJ. No: 09/742,165 

Reply to Final Office Action of July 26, 2006 

Rejections under 35 U.S.C, 103 

in the July 26, 2006 Office Action, claims 1 , 3-6, 8, 11-15, 17-20, 22, 23, and 

25-30 were rejected under 35 U.S,C. 103(a) as being unpatentable over U.S. Pat, 
AppL PubL No. 2002/0129374 ("Freeman") in view of U.S. Pat. No. 6,360,271 
("Schuster"). This rejection is traversed based on the following remarks. 

Before turning to specific claim language, it may be useful to more generally 
describe some of the very significant differences between Applicant's invention and 
the technologies described in Freeman. Applicant's system is effective for mixing 
received streams with video portions that have been streamed over differing 
networks and/or different network paths. This is important where two or more 
sources of video streams are being combined into a single stream. Video time- 
based synchronization requires synchronization in the 100 nanosecond or lower 
range with higher definition requiring even higher levels of synchronization. In part, 
this is because video mixing may involve superimposition of two images or fading 
one into another or cutting a hole and then matting another into the hole (e.g., 
mixing, matting, keying, and the like). For example, a live stream of a news or 
weather anchor may be keyed into a hole created in another live stream (e.g., a 
stream of live action video or weather map or the like) during mixing operations. 
Such mixing may involve mapping video streams on a pixel by pixel basis or even 
fractions of pixels (e.g,, image element in one stream to an image element in 
another stream). Such video mixing is often orders of magnitude more accurate 
than that required to match an audio signal to a video signal. Prior to the invention, 
the problems associated with such mixing of media streams transmitted over the 
Internet and other networks had not been addressed, and one solution was to 
simply display video streams separately (e.g., avoid mixing) or at most try to 
synchronize a sound stream with a video stream but again without mixing, Note 
also that the difference between mere multiplexing of video streams and the mixing 
described by Applicant was explained in detail in Applicant's September 12 T 2005 
Amendment. 
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There are numerous differences between the time-based mixing of received 
signals as taught by Applicant and the teaching of Freeman. Generally, Freeman 
explains in its Background that the methods described are for providing improved 
"seamless switching between compressed digital video signals in a low cost digital 
set top environment" with the video signals being received in a multiplexed signal 
form at the receiver or set top box. As discussed in the Summary, multiple video 
signals are combined by a multiplexer for transmission on desired channels such as 
over a cable or satellite transmission system. According to the Summary, the video 
streams are encoded and time synchronized prior to compression at the single 
source prior to transmittal over a single media or network {such as media 6 in 
Figure 1) {see, for example, para. [0028]), For example, splice points may be 
added to each of the video streams prior to compression and multiplexing to make 
later switching between the separate streams at an end user, with the splice points 
being time synchronized at the source or transmittal point. In other words, Freeman 
is directed to providing an improved way to switch or cut to a different video stream 
and is not concerned at all with mixing two or more video streams into a single 
video stream. Applicant provides methods and systems that combine two or more 
video streams received from differing sources (e.g., over differing networks or 
paths) and with differing transmission times or delays into a single stream in which 
the streams are synchronized {e.g., mapped pixel to pixel in some cases) based on 
transmission time from their sources. In contrast, Freeman teaches controlling the 
source to encode time-synched splice points in video streams prior to multiplexing, 
which requires the use of a single transmission point and control over all sources of 
video received by an end user 

Operation of Freeman's system is explained well with reference to Figures 5 
and 6 beginning with para. [0101]. As discussed in a summary paragraph [0112], 
"seamless switching at the decoder is facilitated at the encoder 312 by time 
synchronizing the signals, time locking the encoders and creating a time gap 340 to 
each of the digital video streams." Figure 5 shows a system provided at a single 
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transmission location and the encoders 312 are time synchronized to allow the 
splice points or time gaps to be provided to allow seamless switching between the 
four video streams that are multiplexed by multiplexer 324 after 
encoding/compression by encoders 312. The multiplexed signals are then 
transmitted by transmitter 328 to an end user (see Figure 7 or Figure 1 ). As can be 
seen, any time synchronization occurs at the transmission source and not at an 
interface where the streams are received after being transmitted over one or more 
transmission networks that will cause them to be subjected to variable transmission 
delays. Also, it can be seen that the video signals are not combined into a single 
mixed stream but are instead simply multiplexed for later selection or switching by 
an end user. 

Other differences between Applicant's invention and Freeman become 
apparent from reading Freeman's Description of the Preferred Embodiments. For 
example, paragraph [0054] discusses the use of multiple data compressors 3 but 
there is no teaching that these compressors utilize different compression standards 
and no discussion is provided of decoding differing standards at the end user's 
system. In fact, Figure 5 shows with encoders 312 that provide signals that 
apparently are compatible since they are multiplexed by multiplexer 324. Beginning 
at para. [0081], Freeman discusses an embodiment that enables seamless flicker- 
free transparent switching between the signals. In para. [0083], the seamless 
switch is described as being "between two or more separate digital video signals," 
which reinforces the construction that the transmitted signals are merely multiplexed 
but separate signals and are not mixed into a single signal prior to transmission. 
Significantly, at the end of para. [0084] and into para. [0085], Freeman teaches that 
seamless splicing between two video streams is supported by switching in "the 
digitally compressed domain thereby eliminating the need to decode two" streams 
at the same time and the compressed video information is placed in a buffer to 
minimize the size of such buffer. In contrast, Applicant shows is Figure 2 that the 
CODEC neutral streams are placed into buffer, and then are combined time- 
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adjusted streams prior to being combined into a single compressed, composite 
stream for transmittal to an end user. 

Turning now to the claim language, claim 1 calls for an input interface that 
receives first and second media streams that each include a streaming video 
portion, A controller determines a "variable transmission delay for the first and 
second media streams from the first and second media sources to the input 
interface and performs the selective retrieving based on the determined variable 
transmission delays." Additionally, the controller mixes the retrieved time-adjusted 
streams "into a composite media stream wherein the first and second time-adjusted 
streams are synchronized based on time." The combined teaching of Freeman and 
Schuster fails to teach or suggest each of these limitations of claim 1 . 

First, Freeman fails to teach the input interface of claim 1 which receives first 
and second media streams that have been transmitted over a communications 
network from "a first and a second media source/ 1 Freeman is cited at paras. [0013] 
to [0016], [0020], and [0048] for teaching such an input interface. However, in 
paras. [0013] to [0016], Freeman discusses an output interface used to transmit 
multiplexed signals over a cable or satellite system for receipt at an end user's 
location. The end user's location is not described as receiving streams from more 
than one media source. In para, [0020], Freeman teaches multiple signals are 
transmitted (in multiplex, compressed form) to the end user's location but again this 
is from a single transmission source, in para. [0048], Freeman states prior systems 
transmitted the signals over separate channels, but the Freeman system instead 
multiplexes the compressed, digital signals to only use a single transmitter 5 and, as 
a result, teaches the use of a single media source and not "a first and a second 
media source" as called for in claim 1 . Hence, Freeman fails to teach at least this 
element of claim 1. 

Second, Freeman fails to teach first and second data buffers for storing the 
data packets of the first and second media streams. As discussed in paras. [0124] 
and [0125], a user selected via host processor 360 a particular one of the 
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multiplexed video signals at the user location and only the selected video is 
"buffered in a standard video buffer and then output for decoding." Also 7 in paras. 
[0084] and [0085], Freeman teaches only one video stream is buffered and then 
decoded when there is enough information "to ensure continuous playback" of the 
selected one of the video signals. Hence, Freeman also fails to show the first and 
second data buffers called for in claim 1 . 

Third, the controller called for in claim 1 is not shown by Freeman because 
Freeman fails to show a controller configured "for selectively retrieving the data 
packets of the first and second media streams" from the buffers to "form a first and 
a second time-adjusted stream." Freeman teaches that only a single stream is 
stored in the end user's buffer and there wouid be no reason to selectively retrieve 
the packets to form time adjusted streams. Further, Freeman cannot teach forming 
first and second streams because only one stream is withdrawn from its buffer and 
then decoded. The portions of Freeman cited in the Office Action all appear to 
occur at the source and discuss time synchronizing with encoders 312 and not after 
received and buffered at the end user's location. For this additional reason, 
Freeman fails to teach each limitation for which it is cited. 

Fourth, the controller "determines a variable transmission delay for the first 
and second media streams from the first and second media sources to the input 
interface and performs the selective retrieving based on the determined variable 
transmission delays." Freeman fails to teach selective retrieving from the end user 
buffer and clearly, not based on a determined variable transmission delay. The 
Office Action agrees that Freeman fails to show such a determination and that 
selective retrieval is performed based on such a determination, Schuster is cited for 
overcoming this deficiency at col. 7, lines 14-67, col. 8, lines 17-56, and col. 12, 
lines 10-67, Schuster teaches the use of synchronized clocks at a transmitting 
device and a receiving device to remove jitter from a single media signal such as by 
providing "substantially the same inter-packet time spacing at the receiving end 
delay period for a sequence of packets." However, there is no discussion of 
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determining such a delay from multiple sources and then using such a transmission 
delay to retrieve media signals from a buffer based on such a determination to 
retrieve time-adjusted signals. As discussed above, Freeman fails to teach such 
selective retrieval as it only shows one selected signal being buffered, and, hence, 
the combination of Schuster and Freeman would only result in the same inter- 
packet time spacing being applied at the user location to the one selected signal as 
was applied to the single media source of Freeman. The combination of the two 
references would not result in the claimed invention. 

Fifth, Freeman faiis to show "mixing" of video streams to provide a composite 
media stream in which the streams are "synchronized based on time," In claim 1, 
the controller further is configured for "mixing the first and second time-adjusted 
streams into a composite media stream wherein the first and second time-adjusted 
streams are synchronized based on time." The Office Action cites Freeman for 
teaching the mixing at paras. [0013] to [0016], para. [0020], para. [0048], paras. 
[0099] and [0100], and paras. [0205] to [0207]. However, as discussed above, 
Freeman only teaches multiplexing of signals and not mixing. The difference 
between these two functions was discussed in the September 2005 Amendment 
submitted by Applicant, Further, the multiplexing occurs at the transmittal source 
such as with multiplexer 324 prior to transmission by transmitter 328. The receiving 
system shown for example in Figure 7 does not include a controller that selectively 
retrieves buffered data packets to form time adjusted streams and then mixes the 
two streams into "a composite media stream wherein the first and second time- 
adjusted streams are synchronized based on time.' 1 Instead, a user selects one of 
the signals in the multiplexed stream and a demux/demodulator 372 processes the 
packets for display via video device 388. For all of these reasons, the combined 
teaching of Freeman and Schuster fails to teach or suggest the system of claim 1 , 
and Applicant requests that the rejection be withdrawn. 

Claims 3-6, 8, and 11-14 depend from claim 1 and are believed allowable 
over Kerr at least for the reasons provided for allowing claim 1 . Claims 3 and 4 are 
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addressed to the two streams having differing compression formats as received and 
handling the streams to allow mixing by decoding into compatible forms. There is 
no discussion of decoding the two streams into formats that are compatible prior to 
storage in buffers in Freeman (or Schuster). Applicant asks Examiner to point to a 
device in Freeman that decodes the video streams that have two differing 
compression formats or to withdraw the rejection. Freeman teaches that a variety 
of compression formats may be used by the encoders 312 but nothing teaches that 
they differ and this would not make sense since their output could not then readily 
be multiplexed for transmittal After receipt at the end user's location, the Freeman 
teaches that only one signal is buffered and it is buffered in a compressed form to 
save memory space. There is no teaching that two streams are decoded and that 
such streams had differing compression formats. Schuster is not cited for providing 
teaching relevant to the limitations of claims 3 and 4. For this additional reason, 
claims 3 and 4 are believed allowable over Freeman and Schuster 

Independent claim 1 5 includes limitations similar to claim 1 . Hence t the 
reasons provided for allowing claim 1 over Freeman and Schuster are believed 
applicable to claim 15, Additionally, claim 15 includes limitations similar to 
dependent claims 3 and 4, and the reasons for allowing claims 3 and 4 over 
Freeman and Schuster are applicable to claim 1 5. Specifically, the Examiner is 
requested to show in Freeman where a decoder decodes two video streams into 
compatible formats (which is useful for facilitating mixing which as discussed with 
reference to claim 1 is not performed by Freeman). For example, Applicant could 
find no discussion of such functions in the description of the decoder 372 shown in 
Freeman's Figure 7, 

Claims 17-19 depend from claim 15 and are believed allowable over 
Freeman and Schuster at least for the reasons provided for claim 1 5, Claim 1 8 
calls for concurrent delivery of time-adjusted first and second streams in a 
composite media stream, and as discussed above for video streams this typically 
involves a resolution or even pixel-by-pixel mapping or the like not shown by 
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Freeman as this reference fails to teach mixing of video streams and instead 
discusses multiplexing at a source. For these additional reasons, claim 18 is 
believed allowable over Freeman and Schuster. 

Independent claim 20 was rejected for substantially for the same reasons as 
provided for claim 1 . Hence, the reasons for allowing claim 1 over Freeman and 
Schuster are believed applicable to claim 20. Additionally, claim 20 cails for the two 
media streams to include data packets from one or more video files and adjusting 
the two streams including "matching the data packets of the first and second media 
streams based on transmittal times from the first and second media sources/' 
Freeman fails to discuss matching the streams it multiplexes based on transmittal 
times from original sources such as elements 300 but instead teaches 
synchronizing the clocks of encoders 312 immediately prior to multiplexing with 
device 324 and then transmitting the multiplexed signal 

Further, in claim 20, a "synchronized media stream" is created "by mixing the 
first and the second media streams" such that the streams are presented 
"concurrently." As discussed with reference to claim 1 , Freeman fails to show any 
mixing of video streams but only discusses multiplexing (and, further, this 
multiplexing is done prior to transmittal over the Freeman network), and as a result, 
the reference cannot anticipate claim 20 because it does not show a stream that 
comprises the mixed first and second streams. Further, claim 20 requires matching 
of data packets of the two streams and this is not shown by Freeman for two video 
streams. The multiplexed streams do not discuss presenting signals concurrently 
but instead a selected signal is buffered and then later encoded for presentation. 
Hence, claim 20 is not anticipated (or even suggested) by Freeman, and Schuster is 
not cited for overcoming these deficiencies in Freeman. Applicant requests that the 
rejection be withdrawn as unsupported by these references. 

Claims 23 and 25-30 depend from claim 20 and are believed allowable over 
Freeman and Schuster for at least the reasons provided for allowing claim 20. 
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Claims 25 and 26 have limitations similar to claims 3 and 4 and are believed 
allowable for the reasons provided for these claims. 

Additionally, the Office Action rejected claims 7, 9, and 10 under 35 LKS.C, 
103(a) as being unpatentable over Freeman and Schuster as applied to claim 5 and 
further in view of U.S. Pat No. 6,934,759 ("Hejna"). Claims 7, 9, and 10 depend 
from claim 1 and are believed allowable over Freeman and Schuster at least for the 
reasons provided for allowing claim 1 over these references, Hejna does not 
overcome the deficiencies of these two references discussed above with reference 
to claim 1 and is not cited for providing such teaching. Therefore, claims 7, 9, and 
10 are believed allowable over the teaching of these three references. 

Conclusions 

In view of all of the above, it is requested that a timely Notice of Allowance be 
issued in this case. 

No fee is believed due with this Amendment. However, any fee deficiency 
associated with this submittal may be charged to Deposit Account No. 50-1 123, 



Respectfully submitted, 


September 18, 2006 


Kent A. LemBke, Reg. No. 44,866 
Hogan & Hartson LLP 
(720) 406-5378 Te 
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